A User-assisted Approach to Multiple Instrument Music Transcription

PhDThe task of automatic music transcription has been studied for several decades and is regarded as an enabling technology for a multitude of applications such as music retrieval and discovery, intelligent music processing and large-scale musicological analyses. It refers to the process of identifying the musical content of a performance and representing it in a symbolic format. Despite its long research history, fully automatic music transcription systems are still error prone and often fail when more complex polyphonic music is analysed. This gives rise to the question in what ways human knowledge can be incorporated in the transcription process. This thesis investigates ways to involve a human user in the transcription process. More specifically, it is investigated how user input can be employed to derive timbre models for the instruments in a music recording, which are employed to obtain instrument-specific (parts-based) transcriptions. A first investigation studies different types of user input in order to derive instrument models by means of a non-negative matrix factorisation framework. The transcription accuracy of the different models is evaluated and a method is proposed that refines the models by allowing each pitch of each instrument to be represented by multiple basis functions. A second study aims at limiting the amount of user input to make the method more applicable in practice. Different methods are considered to estimate missing non-negative basis functions when only a subset of basis functions can be extracted based on the user information. A method is proposed to track the pitches of individual instruments over time by means of a Viterbi framework in which the states at each time frame contain several candidate instrument-pitch combinations. A transition probability is employed that combines three different criteria: the frame-wise reconstruction error of each combination, a pitch continuity measure that favours similar pitches in consecutive frames, and an explicit activity model for each instrument. The method is shown to outperform other state-of-the-art multi-instrument tracking methods. Finally, the extraction of instrument models that include phase information is investigated as a step towards complex matrix decomposition. The phase relations between the partials of harmonic sounds are explored as a time-invariant property that can be employed to form complex-valued basis functions. The application of the model for a user-assisted transcription task is illustrated with a saxophone example.QMU

Automatic music transcription: challenges and future directions

Automatic music transcription is considered by many to be a key enabling technology in music signal processing. However, the performance of transcription systems is still significantly below that of a human expert, and accuracies reported in recent years seem to have reached a limit, although the field is still very active. In this paper we analyse limitations of current methods and identify promising directions for future research. Current transcription methods use general purpose models which are unable to capture the rich diversity found in music signals. One way to overcome the limited performance of transcription systems is to tailor algorithms to specific use-cases. Semi-automatic approaches are another way of achieving a more reliable transcription. Also, the wealth of musical scores and corresponding audio data now available are a rich potential source of training data, via forced alignment of audio to scores, but large scale utilisation of such data has yet to be attempted. Other promising approaches include the integration of information from multiple algorithms and different musical aspects

Aufforstungsprojekte als Bürger*innenwissenschaft in der Oberstufe im Kontext „Bildung für Nachhaltige Entwicklung“ (AProBOS BNE): Projektbericht zur Entwicklung von Unterrichtsmaterialien für einen anwendungs- und methodenorientierten Unterricht

In diesem Beitrag stellen wir den angestrebten doppelten Nutzen unseres Projektes für Bildung und Biotope sowie erste Einblicke und einen Ausblick in die Arbeiten dar, zu denen wir Materialien entwickeln. Dies ist gleichsam ein Übersichtsbeitrag, der das Verständnis einzelner Materialbeiträge späterer Veröffentlichungen rahmen kann. Wir argumentieren zur Relevanz von Aufforstungsprojekten im naturwissenschaftlichen Unterricht der Oberstufe auf zwei Ebenen. Zunächst analysieren wir didaktisch denkend, warum der fachliche Inhalt des Aufforstens für eine oberstufengemäße Aufarbeitung besonders geeignet ist. Anschließend erläutern wir die Bedeutung von Bürger*innenwissenschafts-Projekten (bzw. Citizen-Science-Projekten) zur Aufforstung für den (naturwissenschaftlichen) Unterricht in der Oberstufe. Im dritten Abschnitt stellen wir aus der Sachlogik von Aufforstungsprojekten fachliche und methodische Materialbausteine vor, mit denen Schulen den Einstieg in solche nachhaltigen Aufforstungsprojekte leichter finden können. Es folgen ein Ausblick auf Evaluationsstrategien zur Weiterentwicklung der Materialien und ein kurzer, hoffentlich einladender Einblick in die Ausweitung des Projekts AProBOS BNE zu einem informellen Netzwerk. In this paper, we present the intended dual benefits of our project for education and for biotopes, as well as initial insights and an outlook into the work for which we are developing materials. This overview contribution also provides a frame for understanding individual material contributions in publications to follow. We argue for the relevance of inquiry projects in science education in upper secondary school on two levels. First, we analyze didactically why the subject matter of reforestation is particularly suitable for upper-level instruction. We then explain the relevance of citizen science projects on reforestation for (science) teaching at upper secondary level. Against the background of reforestation projects’ subject logic, we present subject-specific and methodological material modules in the third section, which can support schools in finding their way into such sustainable reforestation projects more easily. The paper closes with an outlook on evaluation strategies for further development of the materials and a short, hopefully inviting, insight into the expansion of the AProBOS BNE project into an informal network

The Code of Protest. Images of Peace in the West German Peace Movements, 1945-1990

The article examines posters produced by the peace movements in the Federal Republic of Germany during the ColdWar, with an analytical focus on the transformation of the iconography of peace in modernity. Was it possible to develop an independent, positive depiction of peace in the context of protests for peace and disarmament? Despite its name, the pictorial selfrepresentation of the campaign ‘Fight against Nuclear Death’ in the late 1950s did not draw on the theme of pending nuclear mass death. The large-scale protest movement in the 1980s against NATO’s 1979 ‘double-track’ decision contrasted female peacefulness with masculine aggression in an emotionally charged pictorial symbolism. At the same time this symbolism marked a break with the pacifist iconographic tradition that had focused on the victims of war. Instead, the movement presented itself with images of demonstrating crowds, as an anticipation of its peaceful ends. Drawing on the concept of asymmetrical communicative ‘codes’ that has been developed in sociological systems theory, the article argues that the iconography of peace in peace movement posters could not develop a genuinely positive vision of peace, since the code of protest can articulate the designation value ‘peace’ only in conjunction with the rejection value ‘war’

Ozone in the Pacific tropical troposphere from ozonesonde observations

Ozone vertical profile measurements obtained from ozonesondes flown at Fiji, Samoa, Tahiti, and the Galapagos are used to characterize ozone in the troposphere over the tropical Pacific. There is a significant seasonal variation at each of these sites. At sites in both the eastern and western Pacific, ozone mixing ratios are greatest at almost all levels in the troposphere during the September‐November season and smallest during March‐May. The vertical profile has a relative maximum at all of the sites in the midtroposphere throughout the year (the largest amounts are usually found near the tropopause). This maximum is particularly pronounced during the September‐November season. On average, throughout the troposphere, the Galapagos has larger ozone amounts than the western Pacific sites. A trajectory climatology is used to identify the major flow regimes that are associated with the characteristic ozone behavior at various altitudes and seasons. The enhanced ozone seen in the midtroposphere during September‐November is associated with flow from the continents. In the western Pacific this flow is usually from southern Africa (although 10‐day trajectories do not always reach the continent) but also may come from Australia and Indonesia. In the Galapagos the ozone peak in the midtroposphere is seen in flow from the South American continent and particularly from northern Brazil. High ozone concentrations within potential source regions and flow characteristics associated with the ozone mixing ratio peaks seen in both the western and eastern Pacific suggest that these enhanced ozone mixing ratios result from biomass burning. In the upper troposphere, low ozone amounts are seen with flow that originates in the convective western Pacific

Multiple light scattering in anisotropic random media

In the last decade Diffusing Wave Spectroscopy (DWS) has emerged as a powerful tool to study turbid media. In this article we develop the formalism to describe light diffusion in general anisotropic turbid media. We give explicit formulas to calculate the diffusion tensor and the dynamic absorption coefficient, measured in DWS experiments. We apply our theory to uniaxial systems, namely nematic liquid crystals, where light is scattered from thermal fluctuations of the local optical axis, called director. We perform a detailed analysis of the two essential diffusion constants, parallel and perpendicular to the director, in terms of Frank elastic constants, dielectric anisotropy, and applied magnetic field. We also point out the relevance of our results to different liquid crystalline systems, such as discotic nematics, smectic-A phases, and polymer liquid crystals. Finally, we show that the dynamic absorption coefficient is the angular average over the inverse viscosity, which governs the dynamics of director fluctuations.Comment: 23 pages, 12 ps figures, to be published in Phys. Rev.

CD152 (CTLA-4) Determines CD4 T Cell Migration In Vitro and In Vivo

BACKGROUND:Migration of antigen-experienced T cells to secondary lymphoid organs and the site of antigenic-challenge is a mandatory prerequisite for the precise functioning of adaptive immune responses. The surface molecule CD152 (CTLA-4) is mostly considered as a negative regulator of T cell activation during immune responses. It is currently unknown whether CD152 can also influence chemokine-driven T cell migration. METHODOLOGY/PRINCIPAL FINDINGS:We analyzed the consequences of CD152 signaling on Th cell migration using chemotaxis assays in vitro and radioactive cell tracking in vivo. We show here that the genetic and serological inactivation of CD152 in Th1 cells reduced migration towards CCL4, CXCL12 and CCL19, but not CXCL9, in a G-protein dependent manner. In addition, retroviral transduction of CD152 cDNA into CD152 negative cells restored Th1 cell migration. Crosslinking of CD152 together with CD3 and CD28 stimulation on activated Th1 cells increased expression of the chemokine receptors CCR5 and CCR7, which in turn enhanced cell migration. Using sensitive liposome technology, we show that mature dendritic cells but not activated B cells were potent at inducing surface CD152 expression and the CD152-mediated migration-enhancing signals. Importantly, migration of CD152 positive Th1 lymphocytes in in vivo experiments increased more than 200% as compared to CD152 negative counterparts showing that indeed CD152 orchestrates specific migration of selected Th1 cells to sites of inflammation and antigenic challenge in vivo. CONCLUSIONS/SIGNIFICANCE:We show here, that CD152 signaling does not just silence cells, but selects individual ones for migration. This novel activity of CD152 adds to the already significant role of CD152 in controlling peripheral immune responses by allowing T cells to localize correctly during infection. It also suggests that interference with CD152 signaling provides a tool for altering the cellular composition at sites of inflammation and antigenic challenge

TRIAD: capturing harmonics with 3D convolutions

This work has been accepted at the 24th International Society for Music Information Retrieval Conference (ISMIR 2023), at Milan, Italy. October 5-9, 2023.Thanks to advancements in deep learning (DL), automatic music transcription (AMT) systems recently outperformed previous ones fully based on manual feature design. Many of these highly capable DL models, however, are computationally expensive. Researchers are moving towards smaller models capable of maintaining state-ofthe- art (SOTA) results by embedding musical knowledge in the network architecture. Existing approaches employ convolutional blocks specifically designed to capture the harmonic structure. These approaches, however, require either large kernels or multiple kernels, with each kernel aiming to capture a different harmonic. We present TriAD, a convolutional block that achieves an unequally distanced dilation over the frequency axis. This allows our method to capture multiple harmonics with a single yet small kernel. We compare TriAD with other methods of capturing harmonics, and we observe that our approach maintains SOTA results while reducing the number of parameters required. We also conduct an ablation study showing that our proposed method effectively relies on harmonic information

Shift-variant nonnegative matrix deconvolution for music transcription

In this paper, we address the task of semi-automatic music transcription in which the user provides prior information about the polyphonic mixture under analysis. We propose a non-negative matrix deconvolution framework for this task that allows instruments to be represented by a different basis function for each fundamental frequency (“shift variance”). Two different types of user input are studied: information about the types of instruments, which enables the use of basis functions from an instrument database, and a manual transcription of a number of notes which enables the template estimation from the data under analysis itself. Experiments are performed on a data set of mixtures of acoustical instruments up to a polyphony of five. The results confirm a significant loss in accuracy when database templates are used and show the superiority of the Kullback-Leibler divergence over the least squares error cost function. Index Terms — semi-automatic music transcription, nonnegative matrix deconvolution, music signal processing 1

A comparison of pitch chroma extraction algorithms

Comunicació presentada a: 19th Sound and Music Computing Conference, celebrat del 5 a 12 de juny de 2022 a Sant-Étienne, França.The pitch chroma is a popular way to represent pitch information in an octave independent way, with applications in automatic chord recognition, cover song identification, audio-to-score alignment, and others. Early chroma extraction algorithms employed expert knowledge to derive pitch chromas from short-time spectra. With the rise of deep learning, the emphasis moved from algorithm design to the structure of the network and the selection of appropriate training data. The approaches perform differently for various types of audio input. We conducted a set of experiments in order to explore the qualitative properties that each algorithm exhibits. These include how the number of concurrent pitches influences the chroma representation, and how noise or unpitched percussion can degrade the performance of the algorithms. We performed a quantitative analysis of various algorithms under these scenarios. The results show that chromas based on deep learning show huge potential, especially when it comes to noise reduction and ignoring non-tonal aspects of the music. However, we also found that some deep learning based chromas fail to accurately detect pitches at lower polyphony levels. We reflect on these results and discuss some paths to improvements for future chroma extraction algorithms